conformance: Optimize mesh weight conformance tests using batch requests #4138
Conversation
The mesh weight conformance tests were executing 500 separate kubectl exec commands with random delays, resulting in very slow test execution. This optimization uses the echo client's --count flag to execute all 500 requests in a single batch, dramatically reducing test time.
k8s-ci-robot
added
release-note-none
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: ciarams87 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
needs-ok-to-test
|
Hi @ciarams87. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/ok-to-test |
k8s-ci-robot
added
ok-to-test
conformance/utils/echo/pod.go
Outdated
| responses := parseMultipleResponses(resp.RawContent) | ||
|
|
||
| if len(responses) != count { | ||
| tlog.Logf(t, "Warning: expected %d responses but got %d", count, len(responses)) |
There was a problem hiding this comment.
can you clarify why this is not an error? if you send 500 requests and gets 499 back, shouldn't this be a problem?
There was a problem hiding this comment.
If this is expected, as I can see you have a "tolerance" configuration below, maybe the tolerance should be part of the MeshPod instance (so each test have a different tolerance) or please add a comment here raising that the caller of RequestBatch expects a tolerance.
There was a problem hiding this comment.
Good catch, thank you! Changed it to return an error. I made it a warning initially due to uncertainty about parsing edge cases, but you're right that missing responses should fail the test immediately. The tolerance is only for validating the weight distribution (±5%), not for handling missing responses.
There was a problem hiding this comment.
thanks! I will come back to this review soon this week!
What this PR does / why we need it:
The mesh weight conformance tests were executing 500 separate kubectl exec commands with random delays, resulting in very slow test execution.
This PR implements an optimization using the echo client's --count flag to execute all 500 requests in a single batch, reducing test time.
Testing:
Ran the mesh tests on istio in a GKE cluster
Performance improvement varies by environment - my tests showed the time went from 60-70s -> 3-4s per test (~95% faster)
Current main:
With the batch requests:
Which issue(s) this PR fixes:
Fixes #4101
Does this PR introduce a user-facing change?: